Quorum based transactionally consistent membership management in distributed storage

ABSTRACT

Systems and methods that restore a failed reconfiguration of nodes in a distributed environment. By analyzing reports from read/write quorums of nodes associated with a configuration, automatic recovery for data partitions can be facilitated. Moreover, a configuration manager component tracks current configurations for replication units and determines whether a reconfiguration is required (e.g., due to node failures, node recovery, replica additions/deletions, replica moves, or replica role changes, and the like.) Reconfigurations of data activated as being replicated from an old configuration to being replicated on a new configuration may be performed in a transactionally consistent manner based on dynamic quorums associated with the new configuration and the old configuration.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/861,448 entitled “Quorum Based Transactionally Consistent MembershipManagement in Distributed Storage” filed Apr. 12, 2013 (Attorney DocketNumber 325870.07), which is a continuation of U.S. patent applicationSer. No. 12/511,525, entitled “Quorum Based Transactionally ConsistentMembership Management in Distributed Storage,” filed Jul. 29, 2009(Attorney Docket Number 325870.01), which claims benefit under 35 U.S.C119(e) of U.S. Provisional Application Ser. No. 61/107,938, filed onOct. 23, 2008. The entireties of the aforementioned applications areincorporated herein by reference.

BACKGROUND

Advances in computer technology (e.g., microprocessor speed, memorycapacity, data transfer bandwidth, software functionality, and the like)have generally contributed to increased computer application in variousindustries. Ever more powerful server systems, which are oftenconfigured as an array of servers, are commonly provided to servicerequests originating from external sources such as the World Wide Web,for example.

As the amount of available electronic data grows, it becomes moreimportant to store such data in a manageable manner that facilitatesuser friendly and quick data searches and retrieval. Today, a commonapproach is to store electronic data in one or more databases. A typicaldatabase can be referred to as an organized collection of informationwith data structured such that a computer program can quickly search andselect desired pieces of data, for example. Moreover, in suchenvironments a federation refers to a group of organizations or serviceproviders that have built trust among each other and enable sharing ofuser identity information amongst themselves.

With the advent of distributed computing models such as web services,there are increased interdependencies among entities such as a ServiceProviders (SP's.) Accordingly, a current trend is to focus oninter-organization and interdependent management of identity informationrather than identity management solutions for internal use. Such can bereferred to as federated identity management. In general, federatedidentity is a distributed computing construct that recognizes thatindividuals move between corporate boundaries at an increasinglyfrequent rate. Practical applications of federated identities arerepresented by large multinational companies that are required to manageseveral heterogeneous systems at the same time.

In such distributed systems, various challenges exist for propermanagement and configuration/reconfiguration of nodes. For example,individual nodes can fail randomly, which can cause data loss whensuitable contingencies are not put into place. Likewise, replicated datais often required to be moved around the system, which can furthercreate reliability issues and consistency problems.

Moreover, reliability issues can further complicate when data related toan over all management of such nodes are subject to loss due to failureof a centralized cache for example.

SUMMARY

Data in a transactional data store may be replicated across manycomputers or other devices acting as nodes in a distributed system, suchas for redundancy or high availability purposes. However, while thedistributed system may provide a high guarantee of availability, theunderlying computers on which the transactional data store is replicatedmay themselves be unreliable, thus making it difficult to maintaintransactional consistency of the transactional data store across thenodes in the distributed system.

A transactional data store configured to be replicated on each node of aset of nodes may be reconfigured to be replicated on each node of adifferent set of nodes.

The reconfiguration from a current configuration to a new configurationmay be done in a transactionally consistent manner partially through theuse of quorums. Each of the current and new configurations may beassociated with at least one quorum, which may be less than or equal tothe number of nodes in the configuration. Reconfiguration operations maybe selectively performed such that they are only performed if a quorumof nodes in the configuration on which the operation will be performedindicate assent to the operation.

The reconfiguration process may involve a number of steps, includingselecting a unique version identifier for the new configuration,updating the transactional data store replicated on nodes in the newconfiguration to a consistent state, deactivating the old configuration,and activating the current configuration. By requiring that progressingfrom one step to the following step of the reconfiguration processinvolves obtaining an acceptance or committal from a quorum of nodes inthe configuration, it is possible to allow the reconfiguration processto be resilient to failures of nodes in a configuration, while stillguaranteeing a level of consistency across the configuration.

The foregoing is a non-limiting summary of the invention, which isdefined by the attached claims.

To the accomplishment of the foregoing and related ends, certainillustrative aspects of the claimed subject matter are described hereinin connection with the following description and the annexed drawings.These aspects are indicative of various ways in which the subject mattermay be practiced, all of which are intended to be within the scope ofthe claimed subject matter. Other advantages and novel features maybecome apparent from the following detailed description when consideredin conjunction with the drawings.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings are not intended to be drawn to scale. In thedrawings, each identical or nearly identical component that isillustrated in various figures is represented by a like numeral. Forpurposes of clarity, not every component may be labeled in everydrawing. In the drawings:

FIG. 1 illustrates a block diagram of a reconstruction componentaccording to a particular aspect of the subject innovation;

FIG. 2 illustrates an exemplary partitioning and a rebuild associatedwith a plurality of nodes according to a particular aspect of thesubject innovation;

FIG. 3 illustrates a block diagram of a system with a configurationcomponent that can be reconstructed in accordance with an aspect of thesubject innovation;

FIG. 4A illustrates a methodology of setting a new configurationaccording to an aspect of the subject innovation;

FIG. 4B is a block diagram of a network in which a reconfigurationprocess may be practiced;

FIG. 4C is a flowchart of a reconfiguration process;

FIG. 5 illustrates an exemplary environment for implementing variousaspects of the subject innovation; and

FIG. 6 is a schematic block diagram of a sample computing environmentthat can be employed for data retrieval according to an aspect of thesubject innovation.

DETAILED DESCRIPTION

The various aspects of the subject innovation are now described withreference to the annexed drawings, wherein like numerals refer to likeor corresponding elements throughout. It should be understood, however,that the drawings and detailed description relating thereto are notintended to limit the claimed subject matter to the particular formdisclosed. Rather, the intention is to cover all modifications,equivalents and alternatives falling within the spirit and scope of theclaimed subject matter.

The inventors have appreciated that, with the advent of distributedcomputing systems, in which data may be replicated across manycomputers, a need exists for mechanisms to reconfigure replicated datain a transactionally consistent manner. The inventors have furtherappreciated that such distributed systems frequently have a need tooffer high availability of the data, even as the underlying computersused to implement the distributed system may themselves occasionallycrash. Transactional data, such as a portion of a database, may bereplicated across a set of nodes, in which there may be a need tomaintain transactional consistency for the data replicated across theset of nodes, even as nodes themselves are unreliable, and may thereforeneed to be added or deleted from the set.

Accordingly, in some embodiments of the invention, a currentconfiguration including a set of nodes, each of which may contain areplica of a transactional data store, may be reconfigured in responseto a reconfiguration request into a new configuration including adifferent set of nodes, all while maintaining transactional consistencyacross the replicated transactional data store.

The transactional consistency is also maintained throughout thereconfiguration process through the use of quorums, in which each of theold and new configurations may be associated with at least one quorum.Reconfiguration operations on the configuration may selectively proceedbased on whether a quorum of nodes in the configuration indicate assentin the operation. The reconfiguration process may involve a number ofsteps, including selecting a unique version identifier for the newconfiguration, updating the transactional data store replicated on nodesin the new configuration to a consistent state, deactivating the oldconfiguration, and activating the current configuration. Assuring thatprogressing from one step to the following step of the reconfigurationprocess requires obtaining an acceptance or committal from a quorumnumber of nodes in the configuration allows the reconfiguration processto be resilient to failures of nodes in a configuration, while stillguaranteeing a level of consistency across the configuration.

Furthermore, other aspects of the system relate to reconstructingdata/information related to reconfiguration of nodes, (the nodes areassociated with a distributed system that implements dynamic quorums ofread/write conditions) via a reconstruction component. In one aspect,the reconstruction component enables storing replicating partial copiesof the information across the distributed system itself. Suchdistributed segments can then be employed to reconstruct content of thecentral management system in a consistent manner. Accordingly, thereconstruction component can reconstruct contents of a centralmanagement component from various locations on the system wherein thecentral management component/configuration component can be treated as acache. Moreover, scalability can be provided via protocol partitioningof the central management component (e.g., employing a same protocol asemployed to make other parts of the system highly available). Likewise,employing a central management component for leadership election for therest of the system allows for flexibility and scale, (typically notafforded if using conventional consensus-based leadership electionalgorithm.)

FIG. 1 illustrates a block diagram for a system 100 that employs areconstruction component 101, which enables reconstructing informationrelated to reconfiguring members of a distributed system. Reconstructioncomponent 101 can further be associated with a leader elector component102 and a cluster configuration component 103, which can facilitatedesignation/operations associated with a primary (e.g., active)configuration manager instance/components.

FIG. 2 illustrates a block diagram for a system 200 that reconstructs aconfiguration manager component using a quorum-based approach accordingto an exemplary aspect.

FIG. 3 illustrates a block diagram of a system 300 that implements aconfiguration manager component 302 in conjunction with a plurality ofnodes as part of a distributed environment such as a ring 310 which canbe reconstructed in accordance with an aspect of the subject innovation.The configuration manager component 302 can reconfigure members of adistributed system of nodes (e.g., servers) form an old configuration toa new configuration, in a transactionally consistent manner byimplementing dynamic quorums based read/write conditions, which mitigatedata loss during such transformation. Such quorum can represent apredetermined number, wherein an addition of the read quorum and thewrite quorum exceeds number of nodes for the configuration (e.g., theread and write quorums of a given configuration overlap.)

FIG. 4A illustrates a related methodology 400 for various stages ofconfiguring a network of nodes, which can be reconstructed in accordancewith a further aspect of the subject innovation. Upon receipt of areconfiguration request, the Reconfiguration Agent on the primaryreplica can initiate deactivation for an old or existing configuration,and supply a further activation of the new configuration (e.g., ensuringthat any transactions whose commits were acknowledged to the client willbe retained by the new configuration; and transactions which had notcommitted or whose commit had not been acknowledged can either becommitted or rolled back). Such can include implementation of fourstages, namely:

Phase 1: Ballot and Catch-Up at 402

During this phase the primary replica proposes a globally unique ID forthe new configuration. The globally unique ID is intended to increasefor each successive configuration. Upon acceptance by a quorum ofreplicas of both the old and new configurations, such an ID isguaranteed to be greater than any previously accepted ID for thispartition. The proposed ID is sent to all replicas in both the old andnew configurations each of which accepts or rejects the ID based onwhether it is greater than any ID they have observed previously.Accordingly, if a replica accepts such ID it can further notify theprimary replica of its latest transaction sequence number and haltsacceptance of new transactions.

Alternatively, if a replica rejects the proposed ID, the primary picks anew higher ID and restarts Phase 1. Once a quorum of replicas from boththe old and new configuration has accepted the proposed ID, the primarydirects the replicas in the new configuration to start catching up sothat the transactional consistency and data safety requirements aremaintained across the reconfiguration. Such can involve a mixture ofcatch-up and transaction rollbacks on individual replicas. Moreover, theprocess is guaranteed to result in a quorum of replicas agreeing on thecurrent state for the content and provides Atomicity, Consistency,Isolation, Durability (ACID) properties across the reconfiguration.Phase 1 can be complete once at least a quorum of replicas in the newconfiguration has been caught up.

Phase 2: Deactivation of Old Configuration at 404

During this phase the primary replica coordinates the deactivation ofthe old configuration. The purpose of deactivation is to guarantee thatit is never possible to find two sets of replicas R1 and R2 such thatR1≠R2 and each replica r1 in R1 claims that configuration C1 is thelatest configuration and R1 forms at least a write quorum of C1 and eachreplica r2 in R2 claims that configuration C2 is the latestconfiguration and R2 forms at least a write quorum of C2; unless C1=C2.Moreover, a deactivation message can be sent to each replica in the oldconfiguration. Each of the replicas can accept the deactivation if itmatches the latest ballot proposal it has accepted. This phase iscomplete when a read quorum of replicas acknowledges the deactivation.

Phase 3: Activation of New Configuration at 406

During such phase the primary replica coordinates the activation of thenew configuration. A purpose of activation is to guarantee that a writequorum of the new configuration knows that the configuration has beenactivated before changes to the content of the replication unit areallowed. Such can ensure that any content changes can be lost only ifquorum of nodes are lost. The activation message can further be sent toeach replica in the new configuration. Each of these replicas can acceptthe activation if it matches the latest ballot proposal it has accepted.Such phase is complete when a write quorum of replicas in the newconfiguration has accepted the activation. At this point the newconfiguration is active and useable.

Phase 4: Commit at 408

Such stage is an optional phase for committing the reconfiguration sinceby the end of Phase 3 the old configuration has been deactivated and thenew configuration has been activated. Yet, such is known only to theprimary replica and from a global outside-of-system perspective.Accordingly, such commit phase distributes this knowledge throughout allinterested parties in the distributed system, namely to each replica inthe old and new configurations. The Configuration Manager may benotified of the completion of reconfiguration at any point after thecompletion of Phase 3 regardless of whether Phase 4 is implemented.

FIGS. 4B and 4C provide more details of the reconfiguration processdescribed in FIG. 4A above. FIG. 4B is an exemplary network 420 in whichthe reconfiguration process may be practiced. Network 420 includes aplurality of nodes communicating over any suitable computercommunications media (e.g., wireless or wired communication, etc.),including a configuration manager 421, which may be similar to theconfiguration manager component discussed above in connection with FIGS.1-3. Network 420 also includes a first set of nodes 422, 424 and 426 ina current (or “old”) configuration 428, and a second set of nodes 442,444 and 446 in a new configuration 448. The first set of nodes and thesecond set of nodes may have any suitable relationship with respect toeach other. For example, the first set of nodes and the second set ofnodes may have one or more nodes in common, they may each comprise thesame set of nodes, or the two sets of nodes may be completely disjoint.The plurality of nodes including the configuration manager 421, thenodes 422 . . . 426, and the nodes 442 . . . 446 may be implemented inany suitable way, including any suitable combination of hardware andsoftware. For example, the configuration manager 421 and the nodes 422 .. . 426 may each be a desktop computer or server computer, using anysuitable computer architecture, and may include an operating system,such as variants of the WINDOWS® operating system developed by MicrosoftCorporation. In some embodiments, one or more nodes, for example, theconfiguration manager 421, may be implemented on a plurality ofcomputers configured to operate as a coherent unit, such as for loadbalancing or redundancy purposes.

Each node may contain or be connected to storage media containing areplication of the distributed data. Each of nodes 422, 424 and 426includes a replica of distributed data, illustrated as replicas 432, 434and 436, respectively. The distributed data may be a component of adistributed storage system. For example, the distributed data replicatedon nodes 422, 424 and 426 may be a portion of a distributed database,such as a data partition in the database. Any suitable distributeddatabase may be used. In some embodiments, variants of the SQL Server®database developed by Microsoft Corporation may be used. Each ofreplicas 432, 434 and 436 may store a plurality of transactions, asdiscussed above.

A reconfiguration operation may change the manner in which data isstored. Rather than being replicated on a first set of nodes,illustrated in FIG. 4B as the current configuration 428, after areconfiguration, the current information will be replicated on a secondset of nodes, illustrated in FIG. 4B as the new configuration 448. Thus,at least by the end of the reconfiguration process, the nodes 442, 444and 446 in the new configuration 428 may also each contain replicas 452,454 and 456, respectively. The reconfiguration process may involveperforming actual movement of data from the current configuration 428 tothe new configuration 448, such as if a node in the new configuration448 does not at first contain a replica of the data partition beingreplicated, or if the replica on a node in the new configuration 448 isout of date. However, in some embodiments or in some scenarios, thenodes in the new configuration may already contain an up-to-date replicaof the data partition, in which case the reconfiguration process mayprimarily involve deactivation of the old configuration and activationof the new configuration.

In some embodiments, the reconfiguration process is initiated by theconfiguration manager 421. The configuration manager 421 may initiate areconfiguration for a number of reasons. For example, reconfigurationscan be triggered by a variety of events, include node or replicafailure, node or replica recovery, replica creation, or otheradministrative actions. The reconfiguration process may be carried outon any suitable node in the network. In some embodiments, theconfiguration manager 421 may be primarily responsible for implementingthe reconfiguration process, while in other embodiments, theconfiguration manager 421 may issue a reconfiguration request to anothernode in the network, which may then carry out the reconfigurationprocess in response to the request. In some embodiments, one node in thenew configuration may be designated as a “primary node,” illustrated inFIG. 4B as “primary reconfiguration agent,” node 442, responsible forcarrying out the reconfiguration process in response to areconfiguration request from the configuration manager 428. Implementingthe reconfiguration process may involve sending messages, such asreconfiguration messages 458 and 459 to and from the primary node 442and other nodes in both the current configuration 428 and the newconfiguration 448.

FIG. 4C illustrates multiple ways in which quorum-based processing canbe used in reconfiguration operations. FIG. 4C illustrates that as partof reconfiguring a distributed database, reconfiguration operations,such as deactivating the old configuration and activating the newconfiguration, are performed. If a quorum of nodes involved in each ofthese reconfiguration operations does not indicate acceptance of amessage indicating that the reconfiguration operation is to beperformed, the reconfiguration operation does not proceed. Conversely,when an appropriate quorum of the nodes involved in the reconfigurationoperation indicates acceptance in response to a reconfiguration messageor command, the reconfiguration operation may proceed.

As illustrated in FIG. 4C, a reconfiguration process may involve movinga data partition from a current (or “old”) configuration of nodes to anew configuration of nodes. While in the discussion that follows, theprocess illustrated in FIG. 4C is assumed to be implemented on theprimary node in the new configuration and is assumed to start inresponse to receiving a reconfiguration request from a configurationmanager, the process of FIG. 4C may be implemented on any suitable nodein the network and may be started based on any suitable event, stimulus,or condition.

In some embodiments, each configuration of a data partition may beassociated with a version identifier, so that a new configuration of adata partition is associated with a different version number. While anysuitable scheme may be chosen for implementing version identifiers, insome embodiments, the version identifier for a new configuration of adata partition should also be higher than any other value for theversion identifier previously or currently associated with that datapartition. Thus, the version identifier for the new configuration shouldbe higher than the version identifier associated with the oldconfiguration. It may also be the intent for the version identifierassociated with the new configuration to be higher than any versionidentifiers associated with any other configurations for that datapartition, including version identifiers associated with failedreconfigurations of that data partition. Though, in other embodiments,other mechanisms may be used to ascertain a current version identifier.

At block 460, the primary node may determine a new value for the versionidentifier associated with the new configuration. In some embodiments,the primary node may select a value higher than the version numberassociated with the old configuration. The primary node may then send amessage containing this proposed value for the new version identifier tonodes in both the new and the old configurations. The primary node maythen wait until it has received responses from at least a quorum ofnodes in the new configuration and a quorum of nodes in the oldconfiguration indicating that the nodes have accepted the proposedversion identifier for the new configuration.

In some embodiments, the quorum associated with the old and newconfigurations will be the same or may be the same percentage of thenodes in the configuration. However, it is not a requirement that thequorums associated with the old and new configurations be the same or bedetermined in the same way. In some embodiments, the primary node maywait for a read quorum of nodes in the old configuration to accept, anda write quorum of nodes in the new configuration to accept.

In some embodiments, the reconfiguration process may not proceed (i.e.,may be suspended) until a read quorum of nodes in the old configurationand a write quorum of nodes in the new configuration accept the proposedversion identifier. Furthermore, in some embodiments, even after a readquorum of nodes in the old configuration and a write quorum of nodes inthe new configuration have accepted the new proposed version identifier,the primary node may still wait for a pre-determined time, or otherevent, to allow slower replicas to respond, and possibly reject theproposed version identifier.

At block 462, the primary node may then check if it has received aresponse from any node in either the new configuration or the oldconfiguration indicating that the node has rejected the proposal. A nodemay reject the proposed version identifier for a number of reasons. Forexample, a node may reject the proposal if it has previously seen aversion identifier associated with the data partition that is higherthan the proposed version identifier. If so, the rejecting node may sendin a rejection response to the primary node the version identifier ithas previously seen.

The process may then proceed to block 464, in which the node on whichthe process is running may determine if it is still the primary node forthe new configuration. In some embodiments, for a number of reasons, theprimary node for a new configuration may change, which may be done inresponse to a command from the configuration manager. If the node onwhich the reconfiguration process is running determines at block 464that it is no longer the primary node, it may, at block 466, abort thereconfiguration process. The reconfiguration process of FIG. 4C may bedone at this point.

Otherwise, if it is determined at block 464 that the node on which thereconfiguration process is running is still the primary, the primarynode may restart the reconfiguration process by returning to block 460.It may then repeat the process discussed above, this time selecting anew proposed version identifier for the new configuration that is higherthan any previously seen version identifier for the configuration,including version identifiers included in a rejection message sent inresponse to a previous proposal.

Otherwise, if the primary node determined at block 462 that no nodesrejected the proposal, then, at block 468, the primary node may updatedata replicated in nodes of the new configuration with the most recentdata replicated in the old configuration. This may be done in anysuitable way. In some embodiments, this operation may be done in twosteps. First, if the primary node does not contain the most recent copyof the replica from the old configuration, then the primary node mayupdate (or “catch-up”) its local replica of the data partition from anode in the old configuration with the most recent data. In someembodiments, the node in the old configuration with the most recent datais selected from among the nodes in the old configuration that acceptedthe version identifier, as discussed in step 460. Determining that anode contains the latest data may be done in any suitable way. In someembodiments, it may be done by checking an identifier associated withthe latest transaction stored in the replica.

Once the primary node has the most recent copy of the replica, in someembodiments, the primary node may then disable user transactions on thedata partition, and instruct the other nodes in the new configuration toupdate their replicas. The primary node may then wait for a write quorumof nodes in the new configuration to be updated.

In some embodiments, updating a replica may involve updating the replicato store at least all the transactions that had been committed by awrite quorum of replicas in the old configuration. In some embodiments,replicas on a write quorum of nodes in the new configuration may beupdated to store a consistent set of transactions. Updating a replicamay thus also include transaction roll-backs, if a particular replica inthe new configuration contains a later transaction than that which wascommitted by a write quorum of nodes in the old configuration. Such asituation could arise if a user transaction was aborted before it wascommitted, such that the transaction was stored on less than a writequorum of nodes in the old configuration. In some embodiments, thereconfiguration process may not proceed (i.e., may be suspended) until awrite quorum of replicas in the new configuration have been updated tostore a consistent set of transactions.

Once a write quorum of nodes in the new configuration has been caughtup, the reconfiguration process may involve sending a reconfigurationmessage to the nodes of the old configuration and/or the newconfiguration, and waiting for a quorum of nodes in the givenconfiguration to accept the message. The reconfiguration message mayinclude, for example, an activation message or a deactivation message.For example, in the example illustrated by FIG. 4C, at block 470, theprimary node may then send a deactivation message to the nodes in theold configuration. The deactivation message may comprise the proposednew configuration, which may include identifiers of the nodes in theproposed new configuration. If a node in the old configuration acceptsthe deactivation message, it may change the state of its current activeconfiguration to being “Deactivated,” and may store the proposed newconfiguration in suitable computer storage on the node, such as on ahard disk.

Each node in the old configuration may also be configured to send back aresponse to the primary node indicating whether the node accepts orrejects the deactivation message. The primary node may then wait for aread quorum of responses from nodes in the old configuration indicatingthat they have accepted the deactivation message.

Next, at block 472, the primary node may determine if a response to thedeactivation message from at least one node in the old configurationindicates that the node has rejected the deactivation message. If anynode has rejected the deactivation message, the reconfiguration processmay proceed to block 464, at which point the reconfiguration processwill either restart or abort, as discussed above.

Otherwise, if it was determined at block 472 that no node in the oldconfiguration rejected the deactivation message, once a read quorum ofnodes in the old configuration have indicated that they have acceptedthe reconfiguration message, at block 474, the primary node may beginthe activation of the new configuration. This may be done by sending anactivation message to all the other nodes in the new configuration. Eachof the other nodes in the new configuration may be configured to send aresponse to the primary node indicating whether the node accepted orrejected the activation message. If a node accepts the activationmessage, it may update its state to an “activated” state, appropriately.In some embodiments, a node that is both part of the old configurationand the new configuration may have already stored locally (e.g., afteraccepting the deactivation message) the state of the new configurationas a proposed configuration, in which case it may, at block 474, updatethe stored state from “proposed” to “activated.” The primary node maythen wait for a write quorum of responses from nodes in the newconfiguration indicating that they have accepted the activation message.

Next, at block 476, the primary node may determine if a response to theactivation message from at least one node in the new configurationindicates that the node has rejected the activation message. If any nodehas rejected the activation message, the reconfiguration process mayproceed to block 464, at which point the reconfiguration process willeither restart or abort, as discussed above.

However, if a write quorum of nodes in the new configuration acceptedthe activation message and no node rejected the activation message, theprimary node may then re-enable transactions on the configuration, andmay also notify the configuration manager, such as the configurationmanager 421, of the committed state of the new configuration. Theprocess may also proceed to block 478, at which point the primary nodemay commit the new configuration. This optional operation may involvesending a configuration commit message containing both the oldconfiguration and the new configuration to all nodes in the networkhaving replicas of the data partition subject to the new configuration.This step may be optional, because in some embodiments, the newconfiguration may become active and complete as soon as a write quorumof nodes in the new configuration has accepted the activation message.The process of FIG. 4C may be done at this point, as the newconfiguration may be fully activated and usable.

As used in this application, the terms “component”, “system”, areintended to refer to a computer-related entity, either hardware, acombination of hardware and software, software, or software inexecution. For example, a component can be, but is not limited to being,a process running on a processor, a processor, an object, an executable,a thread of execution, a program, and/or a computer. By way ofillustration, both an application running on a server and the server canbe a component. One or more components can reside within a processand/or thread of execution, and a component can be localized on onecomputer and/or distributed between two or more computers.

Furthermore, all or portions of the subject innovation can beimplemented as a system, method, apparatus, or article of manufactureusing standard programming and/or engineering techniques to producesoftware, firmware, hardware or any combination thereof to control acomputer to implement the disclosed innovation. For example, computerreadable media can include but are not limited to magnetic storagedevices (e.g., hard disk, floppy disk, magnetic strips . . . ), opticaldisks (e.g., compact disk (CD), digital versatile disk (DVD) . . . ),smart cards, and flash memory devices (e.g., card, stick, key drive . .. ). Additionally it should be appreciated that a carrier wave can beemployed to carry computer-readable electronic data such as those usedin transmitting and receiving electronic mail or in accessing a networksuch as the Internet or a local area network (LAN). Of course, thoseskilled in the art will recognize many modifications may be made to thisconfiguration without departing from the scope or spirit of the claimedsubject matter.

In order to provide a context for the various aspects of the disclosedsubject matter, FIGS. 5 and 6 as well as the following discussion areintended to provide a brief, general description of a suitableenvironment in which the various aspects of the disclosed subject mattermay be implemented. While the subject matter has been described above inthe general context of computer-executable instructions of a computerprogram that runs on a computer and/or computers, those skilled in theart will recognize that the innovation also may be implemented incombination with other program modules. Generally, program modulesinclude routines, programs, components, data structures, and the like,which perform particular tasks and/or implement particular abstract datatypes. Moreover, those skilled in the art will appreciate that theinnovative methods can be practiced with other computer systemconfigurations, including single-processor or multiprocessor computersystems, mini-computing devices, mainframe computers, as well aspersonal computers, hand-held computing devices (e.g., personal digitalassistant (PDA), phone, watch . . . ), microprocessor-based orprogrammable consumer or industrial electronics, and the like. Theillustrated aspects may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. However, some, if not allaspects of the innovation can be practiced on stand-alone computers. Ina distributed computing environment, program modules may be located inboth local and remote memory storage devices.

With reference to FIG. 5, an exemplary environment 510 for implementingvarious aspects of the subject innovation is described that includes acomputer 512. The computer 512 includes a processing unit 514, a systemmemory 516, and a system bus 518. The system bus 518 couples systemcomponents including, but not limited to, the system memory 516 to theprocessing unit 514. The processing unit 514 can be any of variousavailable processors. Dual microprocessors and other multiprocessorarchitectures also can be employed as the processing unit 514.

The system bus 518 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, and/or a local bus using any variety of available busarchitectures including, but not limited to, II-bit bus, IndustrialStandard Architecture (ISA), Micro-Channel Architecture (MSA), ExtendedISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Universal Serial Bus (USB),Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), and Small Computer SystemsInterface (SCSI).

The system memory 516 includes volatile memory 520 and nonvolatilememory 522. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer512, such as during start-up, is stored in nonvolatile memory 522. Forexample, nonvolatile memory 522 can include read only memory (ROM),programmable ROM (PROM), electrically programmable ROM (EPROM),electrically erasable ROM (EEPROM), or flash memory. Volatile memory 520includes random access memory (RAM), which acts as external cachememory. By way of illustration and not limitation, RAM is available inmany forms such as synchronous RAM (SRAM), dynamic RAM (DRAM),synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhancedSDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).

Computer 512 also includes removable/non-removable,volatile/non-volatile computer storage media. FIG. 5 illustrates a diskstorage 524, wherein such disk storage 524 includes, but is not limitedto, devices like a magnetic disk drive, floppy disk drive, tape drive,Jaz drive, Zip drive, LS-60 drive, flash memory card, or memory stick.In addition, disk storage 524 can include storage media separately or incombination with other storage media including, but not limited to, anoptical disk drive such as a compact disk ROM device (CD-ROM), CDrecordable drive (CD-R Drive), CD rewritable drive (CD-RW Drive) or adigital versatile disk ROM drive (DVD-ROM). To facilitate connection ofthe disk storage devices 524 to the system bus 518, a removable ornon-removable interface is typically used such as interface 526.

It is to be appreciated that FIG. 5 describes software that acts as anintermediary between users and the basic computer resources described insuitable operating environment 510. Such software includes an operatingsystem 528. Operating system 528, which can be stored on disk storage524, acts to control and allocate resources of the computer system 512.System applications 530 take advantage of the management of resources byoperating system 528 through program modules 532 and program data 534stored either in system memory 516 or on disk storage 524. It is to beappreciated that various components described herein can be implementedwith various operating systems or combinations of operating systems.

A user enters commands or information into the computer 512 throughinput device(s) 536. Input devices 536 include, but are not limited to,a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 514through the system bus 518 via interface port(s) 538. Interface port(s)538 include, for example, a serial port, a parallel port, a game port,and a universal serial bus (USB). Output device(s) 540 use some of thesame type of ports as input device(s) 536. Thus, for example, a USB portmay be used to provide input to computer 512, and to output informationfrom computer 512 to an output device 540. Output adapter 542 isprovided to illustrate that there are some output devices 540 likemonitors, speakers, and printers, among other output devices 540 thatrequire special adapters. The output adapters 542 include, by way ofillustration and not limitation, video and sound cards that provide ameans of connection between the output device 540 and the system bus518. It should be noted that other devices and/or systems of devicesprovide both input and output capabilities such as remote computer(s)544.

Computer 512 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)544. The remote computer(s) 544 can be a personal computer, a server, arouter, a network PC, a workstation, a microprocessor based appliance, apeer device or other common network node and the like, and typicallyincludes many or all of the elements described relative to computer 512.For purposes of brevity, only a memory storage device 546 is illustratedwith remote computer(s) 544. Remote computer(s) 544 is logicallyconnected to computer 512 through a network interface 548 and thenphysically connected via communication connection 550. Network interface548 encompasses communication networks such as local-area networks (LAN)and wide area networks (WAN). LAN technologies include Fiber DistributedData Interface (FDDI), Copper Distributed Data Interface (CDDI),Ethernet/IEEE 802.3, Token Ring/IEEE 802.5 and the like. WANtechnologies include, but are not limited to, point-to-point links,circuit switching networks like Integrated Services Digital Networks(ISDN) and variations thereon, packet switching networks, and DigitalSubscriber Lines (DSL).

Communication connection(s) 550 refers to the hardware/software employedto connect the network interface 548 to the bus 518. While communicationconnection 550 is shown for illustrative clarity inside computer 512, itcan also be external to computer 512. The hardware/software necessaryfor connection to the network interface 548 includes, for exemplarypurposes only, internal and external technologies such as, modemsincluding regular telephone grade modems, cable modems and DSL modems,ISDN adapters, and Ethernet cards.

FIG. 6 is a schematic block diagram of a sample-computing environment600 that can be employed for implementing nodes as part of a federation,in accordance with an aspect of the subject innovation. The system 600includes one or more client(s) 610. The client(s) 610 can be hardwareand/or software (e.g., threads, processes, computing devices). Thesystem 600 also includes one or more server(s) 630. The server(s) 630can also be hardware and/or software (e.g., threads, processes,computing devices). The servers 630 can house threads to performtransformations by employing the components described herein, forexample. One possible communication between a client 610 and a server630 may be in the form of a data packet adapted to be transmittedbetween two or more computer processes. The system 600 includes acommunication framework 650 that can be employed to facilitatecommunications between the client(s) 610 and the server(s) 630. Theclient(s) 610 are operatively connected to one or more client datastore(s) 660 that can be employed to store information local to theclient(s) 610. Similarly, the server(s) 630 are operatively connected toone or more server data store(s) 640 that can be employed to storeinformation local to the servers 630.

Having thus described several aspects of at least one embodiment of thisinvention, it is to be appreciated that various alterations,modifications, and improvements will readily occur to those skilled inthe art. As an example of a possible variation, in an exemplaryembodiment described above, a quorum of nodes was selected to be amajority of the nodes. Other implementations are possible, with thequorum being either greater or less than a majority of the nodes.Moreover, the quorum may change over time for a configuration as nodesfail or go off-line.

Such alterations, modifications, and improvements are intended to bepart of this disclosure, and are intended to be within the spirit andscope of the invention. Accordingly, the foregoing description anddrawings are by way of example only.

The above-described embodiments of the present invention can beimplemented in any of numerous ways. For example, the embodiments may beimplemented using hardware, software or a combination thereof. Whenimplemented in software, the software code can be executed on anysuitable processor or collection of processors, whether provided in asingle computer or distributed among multiple computers.

Further, it should be appreciated that a computer may be embodied in anyof a number of forms, such as a rack-mounted computer, a desktopcomputer, a laptop computer, or a tablet computer. Additionally, acomputer may be embedded in a device not generally regarded as acomputer but with suitable processing capabilities, including a PersonalDigital Assistant (PDA), a smart phone or any other suitable portable orfixed electronic device.

Also, a computer may have one or more input and output devices. Thesedevices can be used, among other things, to present a user interface.Examples of output devices that can be used to provide a user interfaceinclude printers or display screens for visual presentation of outputand speakers or other sound generating devices for audible presentationof output. Examples of input devices that can be used for a userinterface include keyboards, and pointing devices, such as mice, touchpads, and digitizing tablets. As another example, a computer may receiveinput information through speech recognition or in other audible format.

Such computers may be interconnected by one or more networks in anysuitable form, including as a local area network or a wide area network,such as an enterprise network or the Internet. Such networks may bebased on any suitable technology and may operate according to anysuitable protocol and may include wireless networks, wired networks orfiber optic networks.

Also, the various methods or processes outlined herein may be coded assoftware that is executable on one or more processors that employ anyone of a variety of operating systems or platforms. Additionally, suchsoftware may be written using any of a number of suitable programminglanguages and/or programming or scripting tools, and also may becompiled as executable machine language code or intermediate code thatis executed on a framework or virtual machine.

In this respect, the invention may be embodied as a computer readablemedium (or multiple computer readable media) (e.g., a computer memory,one or more floppy discs, compact discs, optical discs, magnetic tapes,flash memories, circuit configurations in Field Programmable Gate Arraysor other semiconductor devices, or other tangible computer storagemedium) encoded with one or more programs that, when executed on one ormore computers or other processors, perform methods that implement thevarious embodiments of the invention discussed above. The computerreadable medium or media can be transportable, such that the program orprograms stored thereon can be loaded onto one or more differentcomputers or other processors to implement various aspects of thepresent invention as discussed above.

The terms “program” or “software” are used herein in a generic sense torefer to any type of computer code or set of computer-executableinstructions that can be employed to program a computer or otherprocessor to implement various aspects of the present invention asdiscussed above. Additionally, it should be appreciated that accordingto one aspect of this embodiment, one or more computer programs thatwhen executed perform methods of the present invention need not resideon a single computer or processor, but may be distributed in a modularfashion amongst a number of different computers or processors toimplement various aspects of the present invention.

Computer-executable instructions may be in many forms, such as programmodules, executed by one or more computers or other devices. Generally,program modules include routines, programs, objects, components, datastructures, etc. that perform particular tasks or implement particularabstract data types. Typically the functionality of the program modulesmay be combined or distributed as desired in various embodiments.

Also, data structures may be stored in computer-readable media in anysuitable form. For simplicity of illustration, data structures may beshown to have fields that are related through location in the datastructure. Such relationships may likewise be achieved by assigningstorage for the fields with locations in a computer-readable medium thatconveys relationship between the fields. However, any suitable mechanismmay be used to establish a relationship between information in fields ofa data structure, including through the use of pointers, tags or othermechanisms that establish relationship between data elements.

Various aspects of the present invention may be used alone, incombination, or in a variety of arrangements not specifically discussedin the embodiments described in the foregoing and is therefore notlimited in its application to the details and arrangement of componentsset forth in the foregoing description or illustrated in the drawings.For example, aspects described in one embodiment may be combined in anymanner with aspects described in other embodiments.

Also, the invention may be embodied as a method, of which an example hasbeen provided. The acts performed as part of the method may be orderedin any suitable way. Accordingly, embodiments may be constructed inwhich acts are performed in an order different than illustrated, whichmay include performing some acts simultaneously, even though shown assequential acts in illustrative embodiments.

Use of ordinal terms such as “first,” “second,” “third,” etc., in theclaims to modify a claim element does not by itself connote anypriority, precedence, or order of one claim element over another or thetemporal order in which acts of a method are performed, but are usedmerely as labels to distinguish one claim element having a certain namefrom another element having a same name (but for use of the ordinalterm) to distinguish the claim elements.

Also, the phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” or “having,” “containing,” “involving,” andvariations thereof herein, is meant to encompass the items listedthereafter and equivalents thereof as well as additional items.

1-20. (canceled)
 21. A computing device, comprising: a memory and aprocessor that are respectively configured to store and executeinstructions, including instructions for causing the computing device toperform operations for reconfiguring at least a portion of a distributedsystem including a first set of nodes in a first configuration to asecond set of nodes in a second configuration, the at least the portionof the distributed system storing a plurality of transactions, theoperations including: determining an identifier associated with thesecond configuration; updating transactions stored on the secondconfiguration with transactions stored on the first configuration;deactivating the first configuration; and activating the secondconfiguration; wherein each of the proceeding operations is performed inresponse to receiving responses indicating acceptance of that operationfrom at least a quorum number of nodes on which that operation isproposed to be performed.
 22. The computing device of claim 21, wherein:determining the identifier associated with the second configurationcomprises: selecting the identifier associated with the secondconfiguration; sending to each of a plurality of nodes in the firstconfiguration and to each of a plurality of nodes in the secondconfiguration a reconfiguration proposal comprising the selectedidentifier associated with the second configuration; receiving aresponse to the reconfiguration proposal from each node in at least asubset of the first configuration and from each node in at least asubset of the second configuration, wherein each response received froma node contains an indication of an acceptance or a rejection of thereconfiguration proposal; determining whether the received responses tothe reconfiguration proposal indicate acceptance of the reconfigurationproposal by at least the quorum number of nodes; and in response to adetermination that the received responses to the reconfigurationproposal indicate acceptance of the reconfiguration proposal by at leastthe quorum number of nodes, determining that the selected identifier isto be associated with the second configuration.
 23. The computing deviceof claim 21, wherein the operations further comprise: in response to adetermination that received responses to a reconfiguration proposalindicate rejection of the reconfiguration proposal, restarting thereconfiguring.
 24. The computing device of claim 21, wherein: updatingtransactions stored on the second configuration with transactions storedon the first configuration comprises: instructing each node in thesecond configuration to update a replica of the at least a portion ofthe distributed system stored on the node, wherein the updated replicacontains each transaction that has been stored as being committed on atleast a subset of the first configuration, the size of the subset of thefirst configuration representing a first quorum value associated withthe first configuration; and determining whether a number of nodes inthe second configuration that have updated the replica is at least asecond quorum value associated with the second configuration.
 25. Thecomputing device of claim 21, wherein: deactivating the firstconfiguration comprises: sending a deactivation message to each of aplurality of nodes of the first configuration; receiving a response tothe deactivation message from each node in at least a subset of theplurality of nodes of the first configuration, wherein the responses tothe deactivation message contain an indication of an acceptance or arejection of the deactivation; determining whether the receivedresponses to the deactivation message indicate acceptance of thedeactivation by at least the quorum number of nodes; and in response toa determination that the received responses to the deactivation messageindicate acceptance of the deactivation by at least the quorum number ofnodes, deactivating the first configuration.
 26. The computing device ofclaim 25, wherein the operations further comprise: in response to adetermination that a request to deactivate the first configuration wasrejected, restarting the reconfiguring.
 27. The computing device ofclaim 21, wherein: activating the second configuration comprises:sending an activation message to each of a plurality of nodes of thesecond configuration; receiving a response to the activation messagefrom each node in at least a subset of the plurality of nodes of thesecond configuration, wherein the responses to the activation messagescontain an indication of an acceptance or a rejection of the activation;determining whether the received responses to the activation messageindicate acceptance of the activation by at least the quorum number ofnodes; and in response to a determination that the received responses tothe activation message indicate acceptance of the activation by at leastthe quorum number of nodes, activating the second configuration.
 28. Acomputer-readable storage medium, comprising at least one of a memory,disk, disc, or device, that is encoded with computer-executableinstructions that, in response to execution, cause operations forreplicating at least a portion of a distributed system including a firstset of nodes in a first configuration to a second set of nodes in asecond configuration to be performed, the operations including:determining an identifier associated with the second configuration;updating transactions stored on the second configuration withtransactions stored on the first configuration; deactivating the firstconfiguration; and activating the second configuration; wherein each ofthe proceeding operations is performed in response to indications ofacceptance of the particular operation from at least a quorum number ofnodes on which the particular operation is proposed to be performed on.29. The computer-readable storage medium of claim 28, wherein:determining the identifier associated with the second configurationcomprises: selecting the identifier associated with the secondconfiguration; sending to each of a plurality of nodes in the firstconfiguration and to each of a plurality of nodes in the secondconfiguration a reconfiguration proposal comprising the selectedidentifier associated with the second configuration; receiving aresponse to the reconfiguration proposal from each node in at least asubset of the first configuration and from each node in at least asubset of the second configuration, wherein each response received froma node contains an indication of an acceptance or a rejection of thereconfiguration proposal; determining whether the received responses tothe reconfiguration proposal indicate acceptance of the reconfigurationproposal by at least the quorum number of nodes; and in response to adetermination that the received responses to the reconfigurationproposal indicate acceptance of the reconfiguration proposal by at leastthe quorum number of nodes, determining that the selected identifier isto be associated with the second configuration.
 30. Thecomputer-readable storage medium of claim 28, wherein the operationsfurther comprise: in response to a determination that received responsesto a reconfiguration proposal indicate rejection of the reconfigurationproposal, restarting the reconfiguring.
 31. The computer-readablestorage medium of claim 28, wherein updating transactions stored on thesecond configuration with transactions stored on the first configurationcomprises: instructing each node in the second configuration to update areplica of the at least a portion of the distributed system stored onthe node, wherein the updated replica contains each transaction that hasbeen stored as committed on at least a subset of the firstconfiguration, the size of the subset of the first configurationrepresenting a first quorum value associated with the firstconfiguration; and determining whether a number of nodes in the secondconfiguration that have updated the replica is at least a second quorumvalue associated with the second configuration.
 32. Thecomputer-readable storage medium of claim 28, wherein deactivating thefirst configuration comprises: sending a deactivation message to each ofa plurality of nodes of the first configuration; receiving a response tothe deactivation message from each node in at least a subset of theplurality of nodes of the first configuration, wherein the responses tothe deactivation message contain an indication of an acceptance or arejection of the deactivation; determining whether the receivedresponses to the deactivation message indicate acceptance of thedeactivation by at least the quorum number of nodes; and in response toa determination that the received responses to the deactivation messageindicate acceptance of the deactivation by at least the quorum number ofnodes, deactivating the first configuration.
 33. The computer-readablestorage medium of claim 28, wherein: activating the second configurationcomprises: sending an activation message to each of a plurality of nodesof the second configuration; receiving a response to the activationmessage from each node in at least a subset of the plurality of nodes ofthe second configuration, wherein the responses to the activationmessages contain an indication of an acceptance or a rejection of theactivation; determining whether the received responses to the activationmessage indicate acceptance of the activation by at least the quorumnumber of nodes; and in response to a determination that the receivedresponses to the activation message indicate acceptance of theactivation by at least the quorum number of nodes, activating the secondconfiguration.
 34. A method of reconfiguring at least a portion of adistributed system including a first set of nodes in a firstconfiguration to a second set of nodes in a second configuration, themethod comprising operations including: determining an identifierassociated with the second configuration; updating transactions storedon the second configuration with transactions stored on the firstconfiguration; deactivating the first configuration; activating thesecond configuration; and committing the second configuration, whereineach of the proceeding operations is performed in response to acceptanceof that operation by at least a quorum number of nodes on which thatoperation is to be performed.
 35. The method of claim 34, wherein:determining the identifier associated with the second configurationcomprises: selecting the identifier associated with the secondconfiguration; sending to each of a plurality of nodes in the firstconfiguration and to each of a plurality of nodes in the secondconfiguration a reconfiguration proposal comprising the selectedidentifier associated with the second configuration; receiving aresponse to the reconfiguration proposal from each node in at least asubset of the first configuration and from each node in at least asubset of the second configuration, wherein each response received froma node contains an indication of an acceptance or a rejection of thereconfiguration proposal; determining whether the received responses tothe reconfiguration proposal indicate acceptance of the reconfigurationproposal by at least the quorum number of nodes; and in response to adetermination that the received responses to the reconfigurationproposal indicate acceptance of the reconfiguration proposal by at leastthe quorum number of nodes, determining that the selected identifier isto be associated with the second configuration.
 36. The method of claim34, wherein the operations further comprise: in response to adetermination that received responses to a reconfiguration proposalindicate rejection of the reconfiguration proposal, restarting theoperations.
 37. The method of claim 34, wherein: updating transactionsstored on the second configuration with transactions stored on the firstconfiguration comprises: instructing each node in the secondconfiguration to update a replica of the at least a portion of thedistributed system stored on the node, wherein the updated replicacontains each transaction that has been stored as being committed on atleast a subset of the first configuration, the size of the subset of thefirst configuration representing a first quorum value associated withthe first configuration; and determining whether a number of nodes inthe second configuration that have updated the replica is at least asecond quorum value associated with the second configuration.
 38. Themethod of claim 34, wherein: deactivating the first configurationcomprises: sending a deactivation request to each of a plurality ofnodes of the first configuration; receiving a response to thedeactivation request from each node in at least a subset of theplurality of nodes of the first configuration, wherein the responses tothe deactivation request contain an indication of an acceptance or arejection of the deactivation request; determining whether the receivedresponses to the deactivation request indicate acceptance of thedeactivation request by at least the quorum number of nodes; and inresponse to a determination that the received responses to thedeactivation request indicate acceptance of the deactivation request byat least the quorum number of nodes, deactivating the firstconfiguration.
 39. The method of claim 34, wherein the operationsfurther comprise: in response to a determination that deactivationrequest was rejected, restarting the reconfiguring.
 40. The method ofclaim 34, wherein: activating the second configuration comprises:sending an activation request to each of a plurality of nodes of thesecond configuration; receiving a response to the activation requestfrom each node in at least a subset of the plurality of nodes of thesecond configuration, wherein the responses to the activation requestcontain an indication of an acceptance or a rejection of the activationrequest; determining whether the received responses to the activationrequest indicate acceptance of the activation request by at least thequorum number of nodes; and in response to a determination that thereceived responses to the activation request indicate acceptance of theactivation request by at least the quorum number of nodes, activatingthe second configuration.